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with over three-f ourths of the time devoted to lecture, recitation, 
individual seatwork, and videotape viewing of movement events. The DP unit 
was very student-centered, with many procedural and organizational decisions 
left to students; individual roles and responsibilities were not sharply 
defined. Observers in both classes agreed that the DP group exhibited greater 
levels of enthusiasm, involvement, dialogue, questioning, and persistence in 
unit activities. Comparative outcomes data also suggest that the two classes 
developed different understandings about historical events and issues 
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Student Reasoning about Ill-structured Social Problems 
In a Multimedia-supported Learning Environment 

John W. Saye, Auburn University 
Thomas Brush, Arizona State University 

Presented at the annual meeting of the College and University Faculty Assembly of the National 
Council for the Social Studies. Orlando, Florida; November 19, 1999. Portions of this research 
were supported by grants from the Corporation for Public Broadcasting, Auburn University 
College of Education, and Auburn City Schools. 

This paper summarizes findings from a case study exploring high school students' 
responses to a technology-supported, problem-centered U.S. history unit. Problem- 
centered learning is particularly important to the social studies because most social 
educators emphasize civic competency as a central goal (NCSS, 1994). Citizen actors 
must make decisions about a wide variety of ill-structured social problems. Competent 
resolutions of such problems demand civic reasoning: the ability to form persuasive and 
dialectical arguments that demonstrate a genuine, informed consideration of alternatives 
(Newmann, 1990, Parker, Mueller, & Wendling,1989). Efforts to develop competent 
civic problem-solvers have been complicated by obstacles originating within 
organizational structures (e.g., Cuban, 1984; McKee, 1989), teachers (e.g., Saye, 1998; 
Onosko, 1991), and learners. Our project focused on learner difficulties that impede 
competent problem resolution. 
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A team of teacher educators and social studies teachers conceptualized Decision 
Point! (DP), an integrated set of multimedia content resources and investigatory tools for 
exploring the African-American civil rights movement. We wished to explore whether a 
multimedia-supported learning environment might mitigate some of the identified learner 
obstacles to problem-based instruction: (1) failure to become immersed in and persist 
with exploration of the topic (Newmann, 1991; Onosko, 1991; Rossi, 1996), (2) lack of 
domain-specific and metacognitive knowledge necessary for effective problem-solving 
(Parker et al., 1989; Rossi, 1996; Scheurman & Newmann, 1998; VanSickle & Hoge, 
1991; Voss, Greene, Post, & Penner, 1983; Wineburg, 1991), and (3) egocentric 
reasoning that fails to account for competing perspectives (Newmann, 1991; Parker et 
al.,1989). 

Some researchers suggest that the authenticity and richness of multimedia 
learning environments provide qualitatively different experiences that motivate students 
to persist, immerse themselves in the content, encounter diverse perspectives, and 
develop more complex views of issues (Dwyer, 1994; Cognition & Technology Group at 
Vanderbilt, 1992; Kinzie & Sullivan, 1989). Other work suggests that hyperlinked media 
may provide scaffolding support for more disciplined inquiry into ill-structured problems 
(e.g., Hannafin, Land, & Oliver, 1998; Perfetti, Britt, Van Dyke, & Gabrys, 1999). 
However, the research base on technology-rich, student-centered classrooms is limited, 
especially in social studies (Berson, 1996; Ehman, Glenn, Johnson, & White, 1998; Saye, 
1997). Our investigation focused on how participants used the multimedia environment, 
on what meaning this learning experience had for participants, and on how their 
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responses and culminating understandings compared to students who studied the same 
topic without using the multimedia resources. 

Study Design 

Study Participants 

Because we hope to develop a model that can be implemented in typical 
classrooms, we sought to avoid an idealized case setting. The case study teacher had little 
experience in student-centered inquiry, problem-based instruction, or technology use. 
Student participants were enrolled in two sections of a U.S. history course required of all 
1 1 01 graders not taking honors history. The study teacher taught the six-day DP unit to 
one of her classes. We compared that case to a second class studying the same topic with 
the same teacher but using that teacher's preferred expository instructional methods and 
her normal resources. We hoped to gauge how experiences in the DP unit might differ 
from social studies topics as they were usually experienced in this classroom and to 
determine whether student outcomes in the two classes appeared to differ. A diverse 
group of five DP students were selected for follow-up interviews. 

Description of Decision Point Unit Design 

The study teacher identified the topic. She and the other team members 
collaboratively designed the unit problem, the culminating activities, and a jigsaw group 
strategy. In consultation with the teacher, the university team members developed the DP 
hypermedia environment that provided (a) multimedia civil rights content resources in an 
interactive database and (b) scaffolding tools for collecting, analyzing and evaluating 
historical evidence and presenting conclusions (See Appendix A). 
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The unit scenario placed students in the roles of civil rights leaders following the 
death of Martin Luther King, Jr. Student teams investigated the events of the Civil Rights 
Movement from 1954-68 and developed a solution for the unit problem: What strategies 
should be pursued in 1968 to continue the struggle for a more just, equal society? 

Students first joined expert groups to research a particular strategy used in the Movement. 
Then, experts from each group formed decision-making teams who weighed alternative 
solutions and developed a multimedia presentation that used evidence from the database 
to present possible actions and to defend their solution as the wisest choice. Finally, 
students composed individual essays that required them to use unit understandings and 
competencies in a new context (Appendix B). 

The teacher made all decisions about the structure of each day, the creation and 
management of collaborative groups, the degree of teacher direction, and determination 
of unit grades. The university collaborators built scaffolding devices into the DP 
environment and discussed with the teacher possibilities for how they might be used. For 
example, we organized primary documents into conceptually related menus and provided 
an introductory essay to each event that set the documents into a unifying context. A 
second example of support available within the hypermedia environment was a strategic 
scaffold (Hannafin et al., 1998) that posed categorizing questions that an historian might 
use to organize and synthesize evidence about an event. We tried to answer questions that 
the teacher had during unit design and teaching in ways that were informative but not 
directive. We view our collaboration as educative research: the idea that deeper 
knowledge about teaching and learning can arise from dialogues between university 
researchers and teachers (Clark, Moss, Goering, Herter, Leonard, Robbins, Russell, 
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Templin, & Wascha, 1996; Gitlin, 1990). A future report will focus on the teacher’s 
perspective as unit development evolves over the next several years. 

Research Design 

We conceptualized this project as an evaluative case study (Merriam, 1988; 
Patton, 1987). We adopted a hybrid design that incorporated both qualitative and 
quantitative elements in data collection and analysis. Given the theoretical premises that 
undergirded the design of our multimedia environment, we could describe in advance 
some desirable outcomes and develop evaluations to see if they seemed to occur. 
Qualitative methods allowed us to explore plausible explanations for any observed 
effects. 

For both classes we collected comparative data from daily classroom 
observations, parallel pre- and posttests of factual recall knowledge, and end-of-unit 
essays. The essay required students to use unit knowledge to make a judgement about an 
issue related to the unit problem. Building on work by Newmann (1990) and Parker, et al. 
(1989), a four-paragraph essay was scaffolded to assist students in constructing an 
reasoned argument that was both persuasive and dialectical (Appendix B). We defined a 
persuasive reason as one that stated a general principle to support a position or gave a 
specific example from which a general principle could be inferred. Dialectical reasoning 
required that the reasoner suspend judgment and genuinely explore competing points of 
view. 

In addition to comparative data, we collected DP group data from group 
presentation rubrics, attitudinal surveys, and post-unit interviews. The teacher also 
participated in informal daily debriefings and a formal post-unit interview. 
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Findings 

Observations suggested that students in the two classes encountered the unit topic 
in quite different ways. Students in the non-DP class encountered a very teacher-centered 
unit with over three-fourths of the time devoted to lecture, recitation, individual seatwork 
and viewing videotape of movement events. The DP unit was very student-centered. 

After an introductory lecture, the teacher left many procedural and organizational 
decisions to students. Individual roles and responsibilities were not sharply defined. The 
teacher answered questions when asked, but did little to probe student thinking or provide 
intermediate feedback on student progress toward problem solutions. As a result, groups 
groped for direction and appeared disoriented and overwhelmed as they attempted to 
launch each phase of the investigation. As the unit proceeded, groups gradually 
developed strategies for accomplishing tasks, but time and resources were often used 
inefficiently. Although the teacher made adjustments each day and gradually began to 
provide more organizational direction, she continued to provide little input to guide 
students toward more expert thinking about the problem. She acknowledged that she was 
having difficulty conceptualizing her role: “I am going to have to find ways to hold them 
more accountable. I’m not sure how much to let them guide themselves.” 

Despite such organizational problems, observers in both classes agreed that the 
DP group exhibited greater levels of enthusiasm, involvement, dialogue, questioning, and 
persistence in unit activities. Comparative outcomes data also suggested that the two 
classes developed different understandings about historical events and issues involved in 
the phenomenon of the civil rights movement. Differences were evident in factual recall 
and higher order reasoning. 
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Differences in Factual Recall 

Both classes completed pre- and posttests that measured recall of discrete items of 
factual information about the Civil Rights Movement. During the course of the unit, only 
the non-DP group received direct instruction for the information included in the test. The 
teacher was concerned that the DP class would suffer a disadvantage on that measure. 
However, no significant differences were found between groups on either pretests, 1 , 

N=44) = 1.22, p = 0.27, or posttests, % 2 (1, N=44) = 0.00, p = .99, of factual recall. In 

addition to this discrete item test, a portion of the post-unit essay (Paragraph 1) asked 
students to summarize substantive content knowledge that was relevant to deciding the 
social issue posed in the essay. When the need for unit information was contextualized in 
this manner, the DP group recalled significantly more substantive information than the 
non-DP group (See Table 1, Paragraph 1). 

Differences in Higher Order Reasoning 

Paragraphs 2 through 4 of the essay measured persuasive and dialectical 
reasoning about an issue related to the unit problem. Results suggested that the DP group 
was able to apply a more complex understanding of civil rights content towards reasoning 
about social issues than were their peers. Each paragraph yielded a finding about a 
specific competency: 

Persuasive Reasoning. We measured students' general ability to construct a 
persuasive argument and their ability to construct a persuasive argument using unit 
understandings. We judged a reason to be an instance of a persuasive argument if it stated 
a general principle in support of a position or the general principle could be reasonably 
inferred from a discussion of a specific example. 
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Paragraph 2: Students were asked to construct a persuasive written argument 

using information provided in the essay about a topic 
(American Revolution) that was independent of the topic 
studied in the unit. No significant differences were found in the 
general persuasive reasoning abilities of the two groups as 
measured by Paragraph 2 (Table 1). 

Paragraph 3: Students were asked to make a persuasive argument using unit 

content knowledge. Significant differences were found in favor 
of the DP group (Table 1). 

Dialectical Reasoning. We measured students' dialectical reasoning in two ways. 
Students could demonstrate simple dialectical reasoning by fairly representing opposing 
views. Complex dialectical reasoning required students to not only acknowledge a 
counterargument to their chosen position, but to address the reasoning behind that 
argument in defending their choice. 

Paragraphs 2 & 3: Students were generally successful in making arguments that 
were free of bias toward their chosen position ( 23 of 24 in 
non-DP, 18 of 20 in DP group). 

Paragraph 4: In an essay segment measuring complex dialectical reasoning, 

significantly more students in the DP group addressed at least 
one line of support for the counterargument in defending their 
decision about the posed issue, % 2 (1, N=44) = 4.77, g<.05. 

Thirty-five percent of the DP students made such an argument 
compared to eight percent of the non-DP group. 



Reasoning with Social Issues - 9 



Table 1. Summary of Quantitative Analysis of Essay Tests 



Paragraph # 


Data Description 


Group 


N 


M 


SD 


x 2 


1 


# of Factual Statements 


DP 


20 


5.5 


2.4 


4.23* 






non-DP 


24 


3.9 


2.4 




2 


# of Persuasive Arguments 


DP 


20 


1.7 


1.1 


3.46 






non-DP 


24 


1.5 


1.0 




3 


# of Persuasive Arguments 


DP 


20 


2.5 


1.2 


6.08* 






non-DP 


24 


1.6 


1.0 





*p<.05 



DP Student Perspectives 

We cannot claim that our intervention caused the differences evident between 
groups. However, we can combine our observations with the perspectives of participants 
to generate hypotheses about plausible explanations for observed effects. Some students 
in the DP group perceived their experiences with history in this unit to have given them 
different understandings from those gained in previous studies. The great majority of the 
DP group enjoyed the unit (86%) and wished to study other topics in a similar way 
(76%). Interviews provided deeper insight into student perceptions. Most respondents 
endorsed the greater levels of student empowerment and collaborative learning in this 
unit because they believed that these activities helped them construct more complex 
knowledge while also making learning more enjoyable. More importantly, responses 
suggested that interviewees found the DP resources and the problem-driven unit focus: 
(a) provided a more authentic context for encountering historical content, (b) 
personalized history and provoked a more empathetic view of the dilemmas of historical 
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figures, and (c) encouraged meaningful encounters with historical issues that students 
believed would lead to greater retention. 

One student's remarks exemplify elements of all of these themes: 

“You can learn more, maybe, in ... a book, but will you want to know 
more? I mean . . . next year will I remember what I read in this book? 

Probably not, but I will remember those pictures that I saw because they’ll 
stay in my mind. . . [W]hen you say ‘learning,” I can learn the facts, but I 
won’t learn, you know, the experiences. . . and that’s what a lot of people 
remember and those experiences help them learn other things. . . Even 
though you don’t learn more facts, just the fact that you learn more about 
real things, and, like, actuality itself— that just sticks in your mind more 
than facts. . 

Despite these positive findings, multimedia presentations of group resolutions of 
the central unit problem were superficial. Our presentation rubric measured seven 
standards, including accurately identifying relevant information, identifying the reasoning 
behind opposing positions, and defending a clear position with sufficient and appropriate 
evidence. Each standard featured descriptors for scoring at four levels of performance: 
Exemplary (4), Competent (3), Marginal (2), and Poor (1). None of the observers rated 
any of the five group presentations above Marginal on any of the standards closely 
related to complex reasoning or historical understanding. 

In part, this result may stem from the problem of bounded rationality, cognitive 
limitations that cause individuals to construct simplified models of reality (Simon, 1957). 
The individual essays presented students with a more circumscribed problem. In contrast, 




12 



Reasoning with Social Issues -11 



the unit problem demanded consideration of many more variables. Even with the 
advantages of collaboration, the complexity of the problem made model construction 
much more difficult. Furthermore, while the essay task was tightly structured, scaffolding 
supports were either underutilized or missing in student group tasks. Due to the novelty 
of this experience, the teacher was reconceptualizing and adjusting her role as the unit 
unfolded. She regarded her first time teaching this unit as a “learning time” for her. In her 
inaugural effort, definition of group roles and responsibilities and guided discovery were 
minimal. For instance, she did not monitor closely each small group's progress during 
their investigations and provide timely probing questions and feedback to focus their 
thinking. Although they appeared more engaged with their learning than the non-DP 
group, DP students were often confused about the process of inquiry during their 
investigation. Actions and comments by both teacher and students suggested that in this 
initial implementation, all parties struggled to manage the cognitive demands of 
disciplined inquiry. 

Finally, it should be acknowledged that although the DP group essays 
demonstrated more civic reasoning than those of the non-DP group, most did not reach 
the level required for competent decision-making. For example, sixty-five percent of the 
DP class was unable to form a complex dialectical argument. The fact that substantially 
more DP students could construct such an argument than their non-DP peers is 
encouraging, but social educators must be successful with a much greater portion of 
students to meet the goals of civic competence. 
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Conclusions 

In this case study we deliberately chose a classroom where participants had little 
experience with technology-supported, problem-based learning so that we could examine 
the challenges of this type of learning in a typical setting. We hope to follow how this 
unit in this classroom setting evolves over time. Particularly, we hope to learn what 
enhancements to the model unit might be made to assist the development of civic 
competence. 

Study findings should not be generalized. Research in other sites is needed. In 
particular, other studies need to test the potential of these and similar resources in settings 
where teachers are already engaged in problem-based learning with their students. 

This case study suggests promises and difficulties with technology-supported, 
problem-based environments. The greatest contributions of multimedia may be in 
supporting the prerequisites to disciplined thought. Multimedia may afford a more 
authentic context for learning that raises student interest, encourages deeper encounters 
with knowledge, and makes that knowledge more available for application to social 
problems. The realism of such environments may cause students to confront alternative 
perspectives in a more genuine fashion. However, without all of the elements of an 
thoughtful problem-based learning environment in place, it is unrealistic to expect 
engagement to translate into genuine disciplined inquiry into social issues. Expert 
guidance built into a multimedia-supported environment may help students achieve 
modest gains in the disciplined inquiry necessary for addressing social problems 
critically. But study findings highlight the importance of expert guidance by the teacher 
and the need for more grounded investigations of how guided discovery may be 
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effectively implemented to support individual and group decision-making. When 
integrated into a cohesive vision for learning, the strengths of multimedia environments 
might enhance the prospects of building thoughtful classrooms where problems are 
rigorously examined. The potential of embedded scaffolding for supporting teachers in 
the difficult task of guiding thought may make the investigation of problems more 
feasible for more teachers and learners. 
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A ppendix B. Post-Unit Essay 

Comparing Efforts to Bring About Social Change-- 
The American Revolution and the African-American Civil Rights Movement 

In 1776, the American colonists revolted from their mother country, Great Britain. Tensions had 
been growing for some time as the colonists accused Britain of violating their rights. In 1763, colonists 
were banned from living beyond the Appalachian mountains. Settlers already living there lost their 
property. The British Parliament imposed a series of taxes on sugar, tea, paper and other necessities. 
Colonists were not allowed to vote on these taxes. The colonists reacted by writing petitions to the 
government, by boycotting taxed goods and by threatening and assaulting tax collectors. In 1770, five 
colonists were killed in Boston when British troops were called out to quiet a mob. When a group of 
colonists destroyed cargoes of tea on British ships in Boston Harbor in 1773, large numbers of British 
troops were sent to Boston to enforce the law and keep order. Since there was not enough barracks 
space for all of the troops, some colonists were required to house troops in their homes. When troops 
traveled into the countryside in 1775 to seize a supply of weapons and ammunition, shots were fired, 
people were killed on both sides and the American Revolution had begun. 

You have studied efforts in the 1960s by African-Americans to gain their full rights as U.S. 
citizens. Based upon your understanding of the events in that struggle, who do you think had greater 
justification for revolution: American colonists in the 1770s or African-Americans in the 1960s? 



To answer this question, you should write four paragraphs: 



Paragraph 1: Summarize the relevant events of the Civil Rights movement in much the same way that 

the American Revolution is done above. Be careful to avoid in this paragraph revealing 
your position and reasons for your position. That is what paragraph 4 is for. 

Paragraph 2: Make the strongest argument you can that the colonists were more justified in revolting 

than African-Americans would have been. State this position and support it with two or 
three different, good reasons . 



Paragraph 3: Make the strongest argument you can that African-Americans would have been more 

justified in revolting than the colonists. State this position and support it by responding to 
the reasons you gave in your Paragraph 2 argument . 



BOTH PARAGRAPHS 2 & 3 SHOULD READ AS THOUGH THEY ARE THE SIDE YOU ARE 
TRULY TAKING. 



Paragraph 4: Choose which group you believe had the stronger justification for revolution. 

(1) Be sure your conclusion shows that you have considered the arguments against 
your position as well as arguments for your position. 

(2) State what conditions must be present before people are justified in using force to 
bring about change. 

(3) Do you believe those conditions were present for both groups, for neither group or 
for only one group? 





20 



® 

U.S. Department of Education 

Office of Educational Research and Improvement (OERI) 

National Library of Education (NLE) 

Educational Resources Information Center (ERIC) 

REPRODUCTION RELEASE 



(Specific Document) 

I. DOCUMENT IDENTIFICATION: 



Student Reasoning About Ill-structured Social Problems 
Learning Environments 


in Multimedia-supported 


Author(s): John W. Saye; Thomas Brush 


Corporate Source: 




Publication Date: 






1999 





II. REPRODUCTION RELEASE: 

!n order to disseminate as widely as possible timely and significant materials of interest to the educational community, documents announced in the 
monthly abstract journal of the ERIC system, Resources in Education (RIE), are usually made available to users in microfiche, reproduced paper copy, 
and electronic media, and sold through the ERIC Document Reproduction Service (EDRS). Credit is given to the source of each document, and, if 
reproduction release is granted, one of the following notices is affixed to the document. 

If permission is granted to reproduce and disseminate the identified document, please CHECK ONE of the following three options and sign at the bottom 
of the page. 



The sample sticker shown below will be 
affixed to all Level 1 documents 



The sample sticker shown below will be 
affixed to all Level 2A documents 



The sample sticker shown below will be 
affixed to all Level 2B documents 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 

r o\* 




PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL IN 
MICROFICHE. AND IN ELECTRONIC MEDIA 
FOR ERIC COLLECTION SUBSCRIBERS ONLY. 
HAS BEEN GRANTED BY 

0 \* 




PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL IN 
MICROFICHE ONLY HAS BEEN GRANTED BY 






cP 




of 


TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 

1 




TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 

2A 




TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 

2B 



Level 1 



Level 2A 



Level 2B 



t 




i 



i 




Check here for Level 1 release, permitting reproduction 
and dissemination in microfiche or other ERIC archival 
media (e.g., electronic) and paper copy. 



Check here for Level 2A release, permitting reproduction 
and dissemination in microfiche and in electronic media 
for ERIC archival collection subscribers only 



Check here for Level 2B release, permitting 
reproduction and dissemination in microfiche only 



Documents will be processed as indicated provided reproduction quality permits. 

If permission to reproduce is granted, but no box is checked, documents will be processed at Level 1 . 



Sign 

here,-* 

please 




/ hereby grant to the Educational Resources Information Center (ERIC) nonexclusive permission to reproduce and disseminate this document 
as indicated above . Reproduction from the ERIC microfiche or electronic media by persons other than ERIC employees and its system 
contractors requires permission from the copynght holder. Exception is made for non-profit reproduction by libraries and other service agencies 
to satisfy information needs of educators in response to discrete inquiries. 


Signature: / 


Printed Name/PositiorVTitle: 

John W. Saye/Associate Professor 


OrganizatiorVAddress: w 

Auburn University 


Telephone: 

m4) 844-6891 


FAX: 

(3341844-6789 


E-Mail Address: _ . _ 

say e j oMauburn . edu 


n/is/oo 



(over) 



III. DOCUMENT AVAILABILITY INFORMATION (FROM NON-ERIC SOURCE): 

If permission to reproduce is not granted to ERIC, or, if you wish ERIC to cite the availability of the document from another source, please 
provide the following information regarding the availability of the document. (ERIC will not announce a document unless it is publicly 
available, and a dependable source can be specified. Contributors should also be aware that ERIC selection criteria are significantly more 
stringent for documents that cannot be made available through EDRS.) 



Publisher/Distributor: 




Address: 




Price: 





IV. REFERRAL OF ERIC TO COPYRIGHT/REPRODUCTION RIGHTS HOLDER: 

If the right to grant this reproduction release is held by someone other than the addressee, please provide the appropriate name and 
address: 




V. WHERE TO SEND THIS FORM: 



Send this form to the following ERIC Clearinghouse: 

ERIC/CHESS 
2805 E. Tenth Street, #120 
Bloomington, IN 47408 
Attn: Christi Jones 



However, if solicited by the ERIC Facility, or if making an unsolicited contribution to ERIC, return this form (and the document being 
contributed) to: 

ERIC Processing and Reference Facility 
1100 West Street, 2 na Floor 
Laurel, Maryland 20707-3598 

Telephone: 301-497-4080 
Toll Free: 800-799-3742 
FAX: 301-953-0263 
e-mail: ericfac@inet.ed.gov 
WWW: http://ericfac.piccard.csc.com 

ERIC r -088 (Rev. 9/97) 

“cVIOUS VERSIONS OF THIS FORM ARE OBSOLETE. 



